Skip to content

feat(stackable-versioned)!: Integrate with ConversionReviews #1050

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 29 commits into
base: main
Choose a base branch
from

Conversation

sbernauer
Copy link
Member

@sbernauer sbernauer commented May 27, 2025

This PR adds utilities to convert a CRD from one version into a different version. This is tightly integrated with CRD conversions, which send ConversionReviews to registered webhooks and return that review to the K8s apiserver.

TLDR

  • BREAKING: The #[versioned(k8s(skip(merged_crd)))] flag has been removed
  • BREAKING: The version enum used in merged_crd is now suffixed with Version
  • Add new try_convert function to convert objects received via a ConversionReview
  • Add new enable_tracing option to #[versioned(k8s(options(...)))]

Detailed Overview

There are multiple different parts to make this whole process work. All of these parts are automatically generated by the macro and for the user it is a single function call: Foo::try_convert. The individual parts are explained below.

Note

The CRD spec currently does not contain any fields. During the development of the conversion mechanism, this is emitted to more easily focus on the newly generated code. It will eventually contain fields to fully test the implementation.

#[versioned(
    version(name = "v1alpha1"),
    version(name = "v1beta1"),
    version(name = "v1"),
    k8s(group = "foo.stackable.tech")
)]
#[derive(Clone, Debug, CustomResource, Deserialize, Serialize, JsonSchema)]
pub struct FooSpec {}

Updated Version Enum

The generated code tries to abstract away as much of the underlying code as possible. For this use-case, a top-level version enum exists. Before this PR, it only provided an easy way to produce a single, merged CRD based on all defined CRD versions via Foo::merged_crd(Foo::V1). The variants described the available versions and forced users to use well-defined versions instead of &str which can be anything (eg. an invalid version).

This PR changes the inner workings of this enum. Instead of only providing plain variants without any data, each variant now holds the appropriate version of the custom resource.

// Before
pub enum Foo {
    V1Alpha1,
    V1Beta1,
    V1,
}

// After
pub enum Foo {
    V1Alpha1(v1alpha1::Foo),
    V1Beta1(v1beta1::Foo),
    V1(v1::Foo),
}

This change enables us to directly deserialize an object of a custom resource into the enum. This then enables us to generate further (in addition to merged_crd) top-level abstractions on the enum.

With this change it is not possible to use Self to specify the stored apiversion via Foo::merged_crd(). Instead, a new enum for this purpose only is generated.

pub enum FooVersion {
    V1Alpha1,
    V1Beta1,
    V1,
}

impl ::std::fmt::Display for FooVersion {
    fn fmt(
        &self,
        f: &mut ::std::fmt::Formatter<'_>,
    ) -> ::std::result::Result<(), ::std::fmt::Error> {
        f.write_str(self.as_str())
    }
}

impl FooVersion {
    pub fn as_str(&self) -> &str {
        match self {
            V1Alpha1 => "v1alpha1",
            V1Beta1 => "v1beta1",
            V1 => "v1",
        }
    }
}

(De)serialization from/to JSON values

Most (if not all) conversion will be handled automatically by a CRD conversion webhook. Before the K8s apiserver either stores the custom resource in etcd or responds to clients, it will send a ConversionReview to the registered webhook. A list of to be converted objects is provided as a JSON object. This is also represented in Rust code, more specifically a serde_json::Value.

As such, the code needs to be able to turn the JSON object into a Rust representation (in the correct version). Once conversion is performed, the opposite operation needs to be performed: turning the Rust struct back into serialized JSON.

This PR introduces two new (private) methods to do this:

impl Foo {
    fn from_json_value(value: serde_json::Value) -> Result<Self, ()>
    fn into_json_value(self) -> Result<serde_json::Value, ()>
}

Conversion function

Currently, conversion of custom objects (defined by versioned CRDs) is achieved by receiving the ConversionReview via a webhook (which will be run alongside the operator in the future). This ConversionReview is then handed over to the try_convert function. Internally, this will construct strictly typed objects based on the current apiVersion. They are then converted to the desired apiVersion and added to the list of converted objects. If all objects were able to be converted, a successful ConversionReview is returned. Otherwise, a failure is indicated.

impl Foo {
    pub fn try_convert(review: ConversionReview) -> ConversionReview
}

...

Partial expanded code
#[automatically_derived]
pub enum Foo {
    V1Alpha1(v1alpha1::Foo),
    V1Beta1(v1beta1::Foo),
    V1(v1::Foo),
}
#[automatically_derived]
impl Foo {
    #[doc = r" Generates a merged CRD containing all versions and marking `stored_apiversion` as stored."]
    pub fn merged_crd(stored_apiversion:FooVersion) ->  ::std::result::Result< ::k8s_openapi::apiextensions_apiserver::pkg::apis::apiextensions::v1::CustomResourceDefinition, ::kube::core::crd::MergeError>{
        ::kube::core::crd::merge_crds(
            <[_]>::into_vec(
                #[rustc_box]
                alloc::boxed::Box::new([
                    (<v1alpha1::Foo as ::kube::core::CustomResourceExt>::crd()),
                    (<v1beta1::Foo as ::kube::core::CustomResourceExt>::crd()),
                    (<v1::Foo as ::kube::core::CustomResourceExt>::crd()),
                ]),
            ),
            stored_apiversion.as_str(),
        )
    }
    #[doc = "Tries to convert a list of objects of kind [`Foo`] to the desired API version"]
    #[doc = "specified in the [`ConversionReview`][cr]."]
    #[doc = ""]
    #[doc = "The returned [`ConversionReview`][cr] either indicates a success or a failure, which"]
    #[doc = "is handed back to the Kubernetes API server."]
    #[doc = ""]
    #[doc = "[cr]: ::kube::core::conversion::ConversionReview"]
    pub fn try_convert(
        review: ::kube::core::conversion::ConversionReview,
    ) -> ::kube::core::conversion::ConversionReview {
        let request = match ::kube::core::conversion::ConversionRequest::from_review(review) {
            ::std::result::Result::Ok(request) => request,
            ::std::result::Result::Err(err) => {
                return ::kube::core::conversion::ConversionResponse::invalid(
                    ::kube::client::Status {
                        status: Some(::kube::core::response::StatusSummary::Failure),
                        message: err.to_string(),
                        reason: err.to_string(),
                        details: None,
                        code: 400,
                    },
                )
                .into_review();
            }
        };
        let desired_api_version = request.desired_api_version.as_str();
        let response = match Self::convert_objects(request.objects, desired_api_version) {
            ::std::result::Result::Ok(converted_objects) => {
                ::kube::core::conversion::ConversionResponse {
                    result: ::kube::client::Status::success(),
                    types: request.types,
                    uid: request.uid,
                    converted_objects,
                }
            }
            ::std::result::Result::Err(err) => {
                let code = err.http_status_code();
                let message = err.join_errors();
                ::kube::core::conversion::ConversionResponse {
                    result: ::kube::client::Status {
                        status: Some(::kube::core::response::StatusSummary::Failure),
                        message: message.clone(),
                        reason: message,
                        details: None,
                        code,
                    },
                    types: request.types,
                    uid: request.uid,
                    converted_objects: alloc::vec::Vec::new(),
                }
            }
        };
        response.into_review()
    }
    fn convert_objects(
        objects: ::std::vec::Vec<::serde_json::Value>,
        desired_api_version: &str,
    ) -> ::std::result::Result<
        ::std::vec::Vec<::serde_json::Value>,
        ::stackable_versioned::ConvertObjectError,
    > {
        let mut converted_objects = ::std::vec::Vec::with_capacity(objects.len());
        for object in objects {
            let current_object = Self::from_json_value(object.clone())
                .map_err(|source| ::stackable_versioned::ConvertObjectError::Parse { source })?;
            match (current_object, desired_api_version) {
                (Self::V1Alpha1(__sv_foo), "v1beta1") => {
                    let converted: v1beta1::FooSpec = __sv_foo.spec.into();
                    let desired_object = Self::V1Beta1(v1beta1::Foo {
                        metadata: __sv_foo.metadata,
                        spec: converted,
                    });
                    let desired_object = desired_object.into_json_value().map_err(|source| {
                        ::stackable_versioned::ConvertObjectError::Serialize { source }
                    })?;
                    converted_objects.push(desired_object);
                }
                (Self::V1Alpha1(__sv_foo), "v1") => {
                    let converted: v1beta1::FooSpec = __sv_foo.spec.into();
                    let converted: v1::FooSpec = converted.into();
                    let desired_object = Self::V1(v1::Foo {
                        metadata: __sv_foo.metadata,
                        spec: converted,
                    });
                    let desired_object = desired_object.into_json_value().map_err(|source| {
                        ::stackable_versioned::ConvertObjectError::Serialize { source }
                    })?;
                    converted_objects.push(desired_object);
                }
                (Self::V1Beta1(__sv_foo), "v1alpha1") => {
                    let converted: v1alpha1::FooSpec = __sv_foo.spec.into();
                    let desired_object = Self::V1Alpha1(v1alpha1::Foo {
                        metadata: __sv_foo.metadata,
                        spec: converted,
                    });
                    let desired_object = desired_object.into_json_value().map_err(|source| {
                        ::stackable_versioned::ConvertObjectError::Serialize { source }
                    })?;
                    converted_objects.push(desired_object);
                }
                (Self::V1Beta1(__sv_foo), "v1") => {
                    let converted: v1::FooSpec = __sv_foo.spec.into();
                    let desired_object = Self::V1(v1::Foo {
                        metadata: __sv_foo.metadata,
                        spec: converted,
                    });
                    let desired_object = desired_object.into_json_value().map_err(|source| {
                        ::stackable_versioned::ConvertObjectError::Serialize { source }
                    })?;
                    converted_objects.push(desired_object);
                }
                (Self::V1(__sv_foo), "v1alpha1") => {
                    let converted: v1beta1::FooSpec = __sv_foo.spec.into();
                    let converted: v1alpha1::FooSpec = converted.into();
                    let desired_object = Self::V1Alpha1(v1alpha1::Foo {
                        metadata: __sv_foo.metadata,
                        spec: converted,
                    });
                    let desired_object = desired_object.into_json_value().map_err(|source| {
                        ::stackable_versioned::ConvertObjectError::Serialize { source }
                    })?;
                    converted_objects.push(desired_object);
                }
                (Self::V1(__sv_foo), "v1beta1") => {
                    let converted: v1beta1::FooSpec = __sv_foo.spec.into();
                    let desired_object = Self::V1Beta1(v1beta1::Foo {
                        metadata: __sv_foo.metadata,
                        spec: converted,
                    });
                    let desired_object = desired_object.into_json_value().map_err(|source| {
                        ::stackable_versioned::ConvertObjectError::Serialize { source }
                    })?;
                    converted_objects.push(desired_object);
                }
                _ => converted_objects.push(object),
            }
        }
        ::std::result::Result::Ok(converted_objects)
    }
    fn from_json_value(
        value: ::serde_json::Value,
    ) -> ::std::result::Result<Self, ::stackable_versioned::ParseObjectError> {
        let api_version = value
            .get("apiVersion")
            .ok_or_else(|| ::stackable_versioned::ParseObjectError::FieldNotPresent)?
            .as_str()
            .ok_or_else(|| ::stackable_versioned::ParseObjectError::FieldNotStr)?;
        let object = match api_version {
            "foo.stackable.tech/v1alpha1" | "v1alpha1" => {
                let object = ::serde_json::from_value(value).map_err(|source| {
                    ::stackable_versioned::ParseObjectError::Deserialize { source }
                })?;
                Self::V1Alpha1(object)
            }
            "foo.stackable.tech/v1beta1" | "v1beta1" => {
                let object = ::serde_json::from_value(value).map_err(|source| {
                    ::stackable_versioned::ParseObjectError::Deserialize { source }
                })?;
                Self::V1Beta1(object)
            }
            "foo.stackable.tech/v1" | "v1" => {
                let object = ::serde_json::from_value(value).map_err(|source| {
                    ::stackable_versioned::ParseObjectError::Deserialize { source }
                })?;
                Self::V1(object)
            }
            unknown_api_version => {
                return ::std::result::Result::Err(
                    ::stackable_versioned::ParseObjectError::UnknownApiVersion {
                        api_version: unknown_api_version.to_owned(),
                    },
                );
            }
        };
        ::std::result::Result::Ok(object)
    }
    fn into_json_value(self) -> ::std::result::Result<::serde_json::Value, ::serde_json::Error> {
        match self {
            Self::V1Alpha1(__sv_foo) => Ok(::serde_json::to_value(__sv_foo)?),
            Self::V1Beta1(__sv_foo) => Ok(::serde_json::to_value(__sv_foo)?),
            Self::V1(__sv_foo) => Ok(::serde_json::to_value(__sv_foo)?),
        }
    }
}
#[automatically_derived]
pub enum FooVersion {
    V1Alpha1,
    V1Beta1,
    V1,
}
#[automatically_derived]
impl ::std::fmt::Display for FooVersion {
    fn fmt(
        &self,
        f: &mut ::std::fmt::Formatter<'_>,
    ) -> ::std::result::Result<(), ::std::fmt::Error> {
        f.write_str(self.as_str())
    }
}
#[automatically_derived]
impl FooVersion {
    pub fn as_str(&self) -> &str {
        match self {
            V1Alpha1 => "v1alpha1",
            V1Beta1 => "v1beta1",
            V1 => "v1",
        }
    }
}

@sbernauer sbernauer self-assigned this May 28, 2025
@sbernauer sbernauer moved this to Development: In Progress in Stackable Engineering May 28, 2025
@@ -17,6 +17,9 @@ All notable changes to this project will be documented in this file.
- Add `kube_client` crate override to `k8s(crates())` to specify a custom import path. This override
will not be passed to the `#[kube()]` attribute, but will only be available to internal
`#[versioned]` macro code ([#1038]).
- Add `flux-converter`, which adds the `convert` function, which takes a `ConversionReview` and
produces a `ConversionReview` out of it. It creates and uses the needed transitive `.into()` call
chains ([#XXXX]).
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
chains ([#XXXX]).
chains ([#1050]).

@Techassi Techassi moved this from Development: In Progress to Development: In Review in Stackable Engineering Jun 3, 2025
@Techassi Techassi moved this from Development: In Review to Development: In Progress in Stackable Engineering Jun 10, 2025
@Techassi Techassi changed the title feat(stackable-versioned): Add flux-converter feat(stackable-versioned): Integrate with ConversionReviews Jun 11, 2025
@Techassi Techassi marked this pull request as ready for review June 12, 2025 11:53
@Techassi Techassi moved this from Development: In Progress to Development: Waiting for Review in Stackable Engineering Jun 12, 2025
@Techassi Techassi changed the title feat(stackable-versioned): Integrate with ConversionReviews feat(stackable-versioned)!: Integrate with ConversionReviews Jun 12, 2025
Copy link
Member

@NickLarsenNZ NickLarsenNZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left comments and suggestions

@@ -1,5 +1,6 @@
{
"rust-analyzer.cargo.features": "all",
"rust-analyzer.imports.granularity.group": "crate",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to self: check if this is the same in our other rust repos:

  • operator-templating
  • stackablectl
  • docker-images (patchable)
  • containerdebug
  • actions (interu)
  • trino-lb
  • product-config (any reason this can't just be a crate inside operator-rs?)
  • config-utils

There are more, but they are less used.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing to action here, this was just me thinking. I will check it out later.

}
}

impl fmt::Display for Group {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}", self.0)
f.write_str(self)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Way tidier. None of this nameless {} business :p

Comment on lines +4 to +5
pub use bucket::{S3Bucket, S3BucketVersion};
pub use connection::{S3Connection, S3ConnectionVersion};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One day we'll update the operators that use these 😁

@@ -176,4 +161,5 @@ impl ToTokens for KubernetesCrateArguments {
#[derive(Clone, Default, Debug, FromMeta)]
pub struct KubernetesConfigOptions {
pub experimental_conversion_tracking: Flag,
pub enable_tracing: Flag,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this default to?

I think it should be on by default (it still requires a subscriber/exporter for the generated code to actually be used).

/// This function only returns `Some` if it is a struct. Enums cannot be used to define
/// Kubernetes custom resources.
pub fn generate_kubernetes_item(
pub fn generate_kubernetes_code(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could use a doc-comment here

#[snafu(display("encountered unknown object api version {api_version:?}"))]
UnknownApiVersion { api_version: String },

#[snafu(display("failed to deserialize object from json"))]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#[snafu(display("failed to deserialize object from json"))]
#[snafu(display("failed to deserialize object from JSON"))]

Comment on lines +10 to +11
- Add new `try_convert` function to convert objects received via a ConversionReview
- Add new `enable_tracing` option to `#[versioned(k8s(options(...)))]`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Add new `try_convert` function to convert objects received via a ConversionReview
- Add new `enable_tracing` option to `#[versioned(k8s(options(...)))]`
- Add new `try_convert` function to convert objects received via a ConversionReview.
- Add new `enable_tracing` option to `#[versioned(k8s(options(...)))]`.

.iter()
.map(|s| s.ident.to_string())
.collect::<Vec<String>>()
.join("::");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious if there is a syn/syn2 method that already does this 🤔

### `#[versioned(k8s(singular = "..."))]`
Set the singular name. Defaults to lowercased .kind value.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Set the singular name. Defaults to lowercased .kind value.
Set the singular name. Defaults to lowercased `kind` value.

# fn main() {
let merged_crd = Foo::merged_crd(FooVersion::V1Beta1).unwrap();
println!("{}", serde_yaml::to_string(&merged_crd).unwrap());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
println!("{}", serde_yaml::to_string(&merged_crd).unwrap());
println!("{yaml}", yaml = serde_yaml::to_string(&merged_crd).unwrap());

@NickLarsenNZ NickLarsenNZ moved this from Development: Waiting for Review to Development: In Review in Stackable Engineering Jun 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Development: In Review
Development

Successfully merging this pull request may close these issues.

3 participants